Data stream protocol prioritized analysis

ABSTRACT

Methods and apparatuses for performing priority analysis on data transferred in a network. A network processor is connected with a distribution module that distributes network data to multiple memory buffers that are each connected with an analysis processor. Each analysis processor performs protocol analysis tests on the data it receives. When a status signal generated by each memory buffer indicates that the memory buffer is nearing capacity, the corresponding analysis processor may selecting exclude protocol analysis tests based on the priority of those tests. The protocol analyzer is also scalable to include multiple instances of network processors, distribution modules, memory buffers and analysis processors. Each instance is typically interconnected.

CROSS-REFERENCE TO RELATED APPLICATIONS

Not applicable.

BACKGROUND OF THE INVENTION

1. The Field of the Invention

The present invention relates generally to analysis of data transmittedover a communication system. More specifically, the present inventionrelates to expert analysis of network data transmitted a high rate ofspeed.

2. The Relevant Technology

Many data communications systems use a variety of different transmissionmechanisms to enable communication between and among associatedsubsystems. In general, the type of transmission mechanism employed in agiven situation is determined with reference to the particular tasksdesired to be accomplished in connection with those transmissionmechanisms and associated systems. In turn, each transmission mechanismis associated with a particular transmission, or communication, protocolthat defines various parameters concerning the transmission of data inconnection with the transmission mechanism. Such communication protocolscommonly specify, for example, the manner in which data is encoded ontoa transmission signal, the particular physical transmission media to beused with the transmission mechanism, link layers, and other attributesconcerning the transmission of data.

As network data moves from a point of origin to a destination by way ofcommunication links, the network data passes through a variety ofdevices collectively representing multiple protocols and types ofhardware. Typically, each device modifies the network data so that thenetwork data can be transmitted by way of a particular communicationlink. However, modification of the network data in this manner oftencauses errors or other problems with the network data. Such errors mayoccur as the result of various other processes and conditions in thetransmission mechanisms as well. Thus, the various links in acommunications system may be particularly prone to introduce, orcontribute to the introduction of errors in the network data. Moreover,errors and other problems present at one location in the network datastream can cause additional errors or other problems to occur at otherlocations in the network data stream and/or at other points in thecommunications system and associated links.

One approach to the identification, analysis, and resolution of problemsin communications systems involves capturing a portion of the networkdata traffic for review and analysis. In some cases, such data captureis performed in connection with an analyzer that includes varioushardware and software elements configured to capture data fromcommunications links in the communications system, and to present thecaptured data in various formats to a user or technician by way of agraphical user interface or other output device.

Generally, such analyzers capture data traffic in the communicationssystem over a defined period of time, or in connection with theoccurrence of predefined events. Use of the analyzer can allow a networkadministrator to track the progress of selected data as that data movesacross the various links in the communications system. Corrupted oraltered data can then be identified and traced to the problem link(s),or other parts of the communications system. Analyzers can provideuseful results, but it is often the case that employment of typicalprotocol analyzers imposes unacceptable costs in terms of communicationssystem performance and down time. Often, analyzers have been unable toincrease processing speeds to match the increasing rates of datatransfer.

Errors in a communication link can occur at various layers of hardwareand software. Ideally, it is preferred to conduct analysis of everylayer to detect such errors. Example layers of analysis include thephysical layer, the packet layer, the command layer, the applicationlayer, and the network layer. Several different analysis tools have beenproduced to analyze network data so as to detect errors at thesedifferent layers of processing. However, analyzers have generally beenlimited in the number of layers and the amount of data that can beanalyzed.

In addition, at one level of intelligence an analysis tool may be ableto decode an event and present the decoded event to a user ortechnician. Above this level of analysis intelligence is an analysistool that looks at a string of data events that occur over seconds orminutes of time and intelligently analyzes the network data to explainwhat is occurring at a higher level. This may include checking largesequences of packets and primitives using different algorithms and teststo insure that each protocol and application was followed correctly.

Another level of analysis intelligence includes the ability for ananalyzer to look at a higher level of a data communication system andmake sense of the large amount of data transmitted so that the analyzercan indicate to the user or technician what went wrong and also provideinstructions to the user or technician for fixing the problem. However,as these levels of analysis intelligence increase, the amount of dataprocessing power required to perform the analysis also increases.

Another problem with looking at these higher layers is that there can beseveral packets of data making up a transaction between a source and adestination. These data packets can be interleaved with other packets ofdata from different network transactions (e.g., between differentsources and destinations). Thus, to analyze a specific networktransaction, an analyzer must first receive, identify, and associate thedifferent packets from each transaction in order to apply algorithms andother checks to the entire transaction. This becomes even more difficultfor a processor to accomplish as the rate of data transmission, numberof network transactions, and amount of data in each transactionincreases.

BRIEF SUMMARY OF THE INVENTION

The present invention relates to high speed analysis of network data ator approaching real-time speed. According to an embodiment of thepresent invention, a protocol analyzer includes a network processor thatis used to distribute packets of network data to one or more analysisprocessors. Each analysis processor is typically associated with amemory buffer such as a FIFO queue. The protocol analyzer receivesnetwork data representing at least a portion of a data streamtransmitted in a network. The protocol analysis tests are assigned apriority and when a status signal generated by the memory bufferindicates that the memory buffers are nearing capacity, only protocoltests with sufficient priority are performed. In other words, the statussignal causes only selected protocol analysis tests to be performed.

In another embodiment, the protocol analyzer, which includes a networkprocessor, a distribution module, memory buffers and analysisprocessors, can be instantiated multiple times. IN one example, theprotocol analyzer includes two network processors and the network datacan then be supplied to the protocol analyzer from different sources oreach network processor can receive the same data. In one embodiment, alogic device is included to receive all network data. The logic devicethen distributes the packets of the network data to the two instances ofthe protocol analyzer. The instances of the protocol analyzer aretypically connected such that each can utilize the resources of theother and such that results of the protocol analysis can be combined.

These and other advantages and features of the present invention willbecome more fully apparent from the following description and appendedclaims, or may be learned by the practice of the invention as set forthhereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

To further clarify the above and other advantages and features of thepresent invention, a more particular description of the invention willbe rendered by reference to specific embodiments thereof which areillustrated in the appended drawings. It is appreciated that thesedrawings depict only typical embodiments of the invention and aretherefore not to be considered limiting of its scope. The invention willbe described and explained with additional specificity and detailthrough the use of the accompanying drawings in which:

FIG. 1 illustrates example layers of network processing versus relativequantities of data required for the network analysis;

FIG. 2A illustrates a system for demultiplexing and analyzing a networkdata stream according to an example embodiment of the present invention;

FIG. 2B illustrates a system for demultiplexing and analyzing a networkdata stream according to an example embodiment of the present invention;

FIG. 3 is a flow diagram illustrating a method for directing data to ananalysis processor according to an example embodiment of the presentinvention;

FIG. 4A illustrates a network analyzer according to an exampleembodiment of the present invention;

FIG. 4B illustrates a network analyzer having upstream and downstreamlogic device hardware according to an example embodiment of the presentinvention;

FIG. 4C illustrates a typical frame, a modified frame, and a modifiedprimitive according to an example embodiment of the present invention;

FIG. 5 illustrates a storage processor in conjunction with a gigabyteFIFO memory buffer and an analysis processor;

FIG. 6 illustrates a system for analyzing data according to an exampleembodiment of the present invention;

FIG. 7 is a block diagram illustration of a method for analyzing dataaccording to an example embodiment of the present invention;

FIG. 8 is a flow diagram illustrating a method for analyzing a networkdata stream implementing filtering techniques according to an exampleembodiment of the present invention;

FIG. 9 illustrates a network analysis system implementing filteringtechniques according to an example embodiment of the present invention;

FIG. 10 is a block diagram illustrating a method for analyzing dataincluding filtering techniques and demultiplexing of the network data tomultiple FIFO memory buffers coupled to multiple analysis processorsaccording to an example embodiment of the present invention;

FIG. 11 is a block diagram illustrating a method of performing priorityanalysis on a data stream according to an example embodiment of thepresent invention;

FIG. 12 illustrates a system for prioritizing and analyzing datareceived from a network according to an example embodiment of thepresent invention;

FIG. 13 depicts an example priority look-up-table (LUT) listing avariety of tests and analysis algorithms that can be conducted on atransaction or data stream;

FIG. 14 is a block diagram illustrating a method for analyzing datacombining at least filtering techniques, priority analysis techniques,and demultiplexing of a data to multiple analysis processors aspects ofexample embodiments of the present invention;

FIG. 15 illustrates a system for analyzing network data including atransaction distribution module implementing filtering and priorityanalysis techniques according to an example embodiment of the presentinvention;

FIG. 16 illustrates a network processing system where the networkprocessor includes two network data inputs according to an exampleembodiment of the present invention;

FIG. 17 illustrates an example embodiment of the present invention wherethe network analysis system includes multiple inputs for receivingnetwork data;

FIG. 18 illustrates an example embodiment of the present invention wheremultiple network processors have been implemented;

FIG. 19 illustrates an example embodiment of the present inventionimplementing multiple network processors and multiple network datadistribution modules;

FIG. 20 illustrates a system for analyzing a network data streamimplementing a front-end programmable logic device for diverting networkdata between two network processors according to an example embodimentof the present invention;

FIG. 21 illustrates a network analysis system including a single networkprocessor that distributes network data between two distribution modulesoriented in parallel with respect to the network processor; and

FIG. 22 illustrates a network analysis system including a single networkprocessor that distributes network data between two distribution modulesoriented in series with respect to the network processor.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention is directed toward the analysis of data in highspeed data transmission systems. The principles of the present inventionare described with reference to the attached drawings to illustrate thestructure and operation of example embodiments used to implement thepresent invention. Using the diagrams and description in this manner topresent the invention should not be construed as limiting its scope.Additional features and advantages of the invention will in part beobvious from the description, including the claims, or may be learned bythe practice of the invention. Descriptions of well-known components andprocessing techniques are omitted so as not to unnecessarily obscure theinvention in detail.

An apparatus for analyzing a data stream can analyze a variety ofdifferent layers of the network data transmission to locate errorscaused by different mechanisms and processes. FIG. 1 depicts examplelayers of processing 100 versus relative quantities of data required foranalysis 110. An analysis tool can perform analysis at several differentlayers 100 of a transmitted data stream that may require analysis ofseveral different quantities of data 110 for analysis. For example, oneanalysis tool 140 might look at the physical and packet layers of a datatransmission, while another analysis tool 130 might look at the commandand application layers.

Different layers 100 may identify errors in different mechanisms andprocesses of a network. For example, the physical layer (PHY) mayaddress hardware errors that are associated with electronic signals. Thepacket layer (PKT) may be more directed toward errors in both hardwareand firmware mechanisms and processes. The command layer (CMD) may bemore directed toward detecting errors in groups of packets of data andoperating system errors. An application layer (APP) may be moreconcerned with detecting errors at the application protocol level andmore intelligent interpretation of data may be required. Finally, at thenetwork layer (NETWORK) there are applications and links workingsimultaneously and effects may not be readily identifiable at the causeof the error, and an error may need to be traced from where it isidentified to the location of its cause.

Often, in order to analyze a data stream at a higher layer, a largerportion of the network data stream may need to be analyzed at one time.For example, to analyze a data stream at the physical and packet layer,only a single packet may need to be analyzed at a time. However, at thecommand, application, and network layers, multiple packets of datarelated to entire transactions may need to be analyzed at a time todetect errors. A transaction can be defined as a task, exchange, orcommand involving one or more packet transmissions. To achieve analysisof such higher layers often requires additional processing requiringadditional processing power. Some embodiments of the invention relate toapparatuses and methods for expert data analysis of one or more layersfor errors at, or approaching, real-time speed. Real-time speed can bedefined as a speed that can keep up with the incoming trafficindefinitely in a controlled manner without skipping portions of thenetwork data in order to catch up. Some embodiments of the presentinvention can operate at, or near, real-time speed.

Some embodiments of the present invention also relate to performinganalysis of network data at various layers of analysis. The term‘network data’ refers to a transmission, packet, primitive, data, andany other information transferred in a communications link, data link,wireless link, optical link, copper link, Fibre channel link, Ethernetlink, or other link of a data or communications system. For example,some advantageous aspects of the present application that can becombined in several different configurations, sequences, andaccomplished using a variety of apparatuses and processes include: (1)demultiplexing of network data so that the network data can be directedto and/or analyzed by multiple analysis processors, (2) distributing apiece of network data, or portions of network data, across multipleprocessors for network analysis, (3) filtering network data so as toreduce the amount of processing power required by excluding network datasuch as repetitive data or data with known analysis results from furtheranalysis, (4) prioritizing different analysis tests and algorithms sothat less critical tests, tests that have already been conducted, testswith known results, and/or other tests can be excluded for the sake ofmore critical tests, and (5) scaling various aspects of the presentinvention so as to remove bottlenecks in network analysis apparatuses.

1. Demultiplexing Network Data for Analysis

Referring now to FIG. 2A, a system 200 for demultiplexing and analyzinga network data stream is shown according to an example embodiment of thepresent invention. The system 200 for demultiplexing and analyzing adata stream can be placed at any point along a transmission link 205between a data transmission source 210 and a data transmissiondestination 215. A communication system including the source 210 and thedestination 215 can comprise a variety of different communication links,systems, and devices conforming to any number of different communicationprotocols.

The signals transmitted between the source 210 and destination 215 arereceived by the analyzer 200 using a physical connection 220 coupled tothe transmission link 205. The physical connection 220 can include atapping apparatus that allows the network data stream to continue on tothe destination component 215 without disrupting the transmission ofdata. Use and manufacture of tapping apparatuses are generally wellknown to one of ordinary skill in the art and any appropriate tappingdevice can be used according to the present invention.

The physical connection 220 can be part of the network analysis system200 depicted in FIG. 2, or can be part of any of the embodiments of thepresent invention described herein. The physical connection 220 may alsobe a separate distinct apparatus coupled to embodiments of the presentinvention in an appropriate manner (e.g., as shown in FIG. 2) to providenetwork data for analysis. The physical connection 220 can produce acopy of at least a portion of the network data stream and forward thecopy representing at least a portion of the network data stream to theanalysis system 200 without disrupting the transmission of data betweenthe source 210 and destination 215. The copy of the network data streamcan then be routed to several analysis processors 260 a-n for analysisof the mechanisms and processes involved in the network datatransmission processes.

As shown, the copy of the network data stream is received by a networkprocessor 230. The network processor 230 can be programmable and caninclude computer executable instructions and additional internal orexternal processors and memory as needed to identify and manipulate thenetwork data in the copy of the network data stream, and to communicatecontrol signals to a distribution module 240. The network processor 230can be any device that keeps track of transactions. For example, thenetwork processor 230 can be a FPGA, an EZ-chip, a microprocessor, orother logic device, but is not limited to processors that executesoftware or firmware. The control signals can be any appropriateinstructions, signal, or code capable of providing instructions to thedistribution module 240 for directing the network data to any of theanalysis processors 260 a-n. The network processor 230 can identifydifferent portions of the network data stream by transaction, by source,by destination, by protocol, by data type, or by any other network ordata attribute and direct the appropriate portions of the network datastream to any of the analysis processors 260 a-n based on theidentification.

The distribution module 240 receives the network data stream from thenetwork processor 230 and routes it to any of its several possibleoutputs according to the control signal received from the networkprocessor 230. While the components of the network analysis system 200,such as the network processor 230 and the distribution module 240, areshown as distinct devices it should be appreciated that any of thecomponents shown in any of the embodiments described herein, such as thenetwork processor 230 and distribution module 240, can be combined intoan integrated device design or broken into additional distinctcomponents for accomplishing the described functions according toembodiments of the present invention. The outputs of the distributionmodule 240 that do not receive the network data are typically held inthe inactive state or open-circuited, depending on the type ofdistribution module 240. The outputs of the distribution module 240 canall be held in the inactive state or open circuited in the instance thatthe network processor 230 determines that the network data should not besent to any of the analysis processors 260 a-n.

Upon routing the network data to a particular output of the distribution250 a-n. The memory buffers 250 a-n can be any appropriate type ofmemory buffer. For example, the memory buffers 250 a-n can befirst-in-first-out (FIFO) memory buffers coupled to the analysisprocessors 260 a-n.

A FIFO memory buffer allows received data to “fall through” to itsoutput queue with only a small delay. In one embodiment, input andoutput from the FIFO are controlled by separate clocks, and the FIFOkeeps track of what data has entered and what data has been removed. Assuch, data is not lost if an analysis processor connected to the FIFO isnot ready for each portion of the network data stream (e.g., packet ofnetwork data) as it is received by the FIFO so long as the FIFO is notallowed to fill up completely.

Thus, according to the example embodiment shown in FIG. 2A, the routingof the network data stream to each of the memory buffers 250 a-n coupledto the analysis processors 260 a-n is controlled by the networkprocessor 230 such that the processing of data is distributed betweenthe analysis processors 260 a-n. Moreover, the analysis processing ofthe network data can be distributed between the different analysisprocessors 260 a-n such that multiple pieces of data belonging tospecific transactions can be directed toward a particular analysisprocessor (e.g., 260 a). In this manner, the different transactionsincluded in the network data stream can be sorted to some extent priorto reaching the analysis processors 260 a-n.

Each analysis processor 260 a-n can also include, or be coupled to,memory (e.g., a hard disk drive (HDD)) for storage of data and storageof any results of the analysis conducted. Each analysis processor 260a-n can also be coupled to user input devices such as a keyboard andoutput such as a display or a printer. The analysis processors can alsobe incorporated into higher level data processing and storage systems aswell as networks of computers. Additional hardware and/or processors canalso be implemented as needed to accomplish each task.

Several different devices can be implemented to perform the tasks andprocesses described herein. Referring to FIG. 2B, a system 202 fordemultiplexing and analyzing a network data stream is shown according toan example embodiment of the present invention. The system 202 fordemultiplexing and analyzing a data stream is similar to that shown inFIG. 2A, except that a field programmable gate array (FPGA) 245, orother logic device, is used in conjunction with the network processor230 to distribute, demultiplex, sort, and direct the network data or thepackets of data to the analysis processors 260 a-n. The system 202 canbe adapted to analyze one or more layers as illustrated in FIG. 1. TheFPGA is an example of a distribution module. Another example is ademultiplexer or a router chip.

Signals transmitted between the source 210 and destination 215 arereceived by the analysis system 202 using the physical connection 220coupled to the transmission link 205. A copy of the network data streamis received by the network processor 230. The network processor 230 canbe programmable and can include computer executable instructions, andadditional internal or external processors and memory as needed toidentify and manipulate the network data. The network processor canprovide any appropriate signal capable of providing instructions to theFPGA 245 for directing the network data to any of the analysisprocessors 260 a-n. For example, according to the embodiment depicted inFIG. 2B, the network processor 230 can insert fields into the networkdata that indicate to the FPGA 245 which analysis processor 260 a-n toroute the network data to. The network processor 230 can identifydifferent portions of the network data stream by transaction, by source,by destination, by protocol, by data type, or by any other network ornetwork data attribute and direct the appropriate portions of thenetwork data stream to any of the analysis processors 260 a-n based onthe identification. An identification can be inserted into the networkdata at particular points that are received and recognized by the FPGA245 and provide instructions to the FPGA 245 for routing the networkdata to any of the analysis processors 260 a-n.

The FPGA 245 receives the network data stream from the network processor230 and routes it to any of its several possible outputs according tothe instructions received from the network processor 230 along with thenetwork data. Upon routing the network data to a particular output ofthe FPGA 245, the network data is received within at least one ofseveral memory buffers 250 a-n. The routing of the network data streamto each of the memory buffers 250 a-n coupled to the FPGA 245 can becontrolled by the network processor 230 and carried out by the FPGA 245.

In some instances, for a processor to analyze a transaction at a higherlayer the processor may need to receive the entire transaction prior toconducting the analysis. For example, FIG. 3, illustrates an examplemethod for directing data to an analysis processor. Network datarepresenting data transmitted in a network is received (300). Aparticular transaction to which the packet or primitive belongs isidentified (305). The packet or primitive is assigned identification(310).

It is next determined whether the transaction has been assigned to aparticular analysis processor for analysis (315). There can be anynumber of processors for analyzing the various transactions communicatedacross a network. In the case that the transaction has not yet beenassigned to a particular analysis processor for analysis, load balancingcan be performed. In load balancing, the amount of data contained withineach memory buffer (see, e.g., buffers 250 a-n in FIG. 2 b) connected toa processor can be compared (320) to determine an appropriate analysisprocessor to assign the particular transaction. For example, anappropriate analysis processor can be determined on the basis of theleast amount of data held within a corresponding memory buffer so thatthe total analysis processing burden can be evenly distributed acrossthe processors. Certain processors can also be designated for particulartypes of analysis, network data types, or transactions, and packets andprimitives can be directed to an appropriate processor on the basis of adesired type of analysis, or for any other reason.

The transaction can be assigned to an appropriate analysis processor(325), and the network data belonging to the transaction is sent to thecorresponding memory buffer (330). The desired analysis is in turnconducted on the data, primitives, or packets of data making up atransaction (335) as the case may be. The analysis can include analysisof the network data according to any of the various layers of analysisdiscussed above with reference to FIG. 1 for example. After the analysisis conducted, results of the analysis can be stored in a HDD, orpresented to a user (340), for example.

Apparatuses for practicing methods of demultiplexing and analyzingnetwork data for errors can include various components, processes, andconfigurations according to different embodiments of the presentinvention. For example, referring now to FIG. 4A, a network analyzer 400is shown according to an example embodiment of the present invention. Anetwork 405 includes a host 410 and a target 415 in bidirectionalcommunication. The network 405 can include several hosts and targetswith packets of data relating to different transactions interleaved asis common in the transmission of data across communication networks.

The network analyzer 400 can receive network data from a physicalconnection 420 placed on a network link for producing a copy of thenetwork data representing the network data stream transmitted across thenetwork link, and forwarding the copy of the network data to a networkprocessor 425. The network processor 425 can be any programmable networkprocessor and can include multiple processors for executing logic toperform the described tasks. The network processor can also includeinternal and/or external memory devices for storing and accessing data.For example, according to an example embodiment of the presentinvention, the network processor 425 can be a general purposeprogrammable network processor such as EZchip NP-1c, which is made byEZchip. An EZchip processor is an example of a network processor thatmay be programmed to provide the functions described herein at a rate ofspeed sufficient for many example embodiments of the present invention.

The EZchip NP-1c is a 10-Gigabit full-duplex network processor providingfully programmable 7-layer packet classification, modification,forwarding and policing at wire speed. The 7-layer deep packetprocessing can provide support for advanced services and applicationssuch as Network Address Translation (NAT), firewall, Virtual PrivateNetworks (VPN), load balancing, storage and traffic analysis in additionto mainstream Layer 2-4 switching and routing applications.

In operation, the network processor 425 receives the network data streamincluding a data packet. A S/D/Q parser 430 extracts identificationfields from the packet of data. For example, the parser 430 can beembodied as hardware and executable logic configured to extract fieldssuch as source, destination, and Q tag (S/D/Q) information from a packetof data. The S/D/Q information can relate to the source and destinationof the transaction to which the packet of data belongs as well as anidentification number (Q tag) assigned by the transmission system to theparticular packet. The S/D/Q information is sent to a S/D/Qlook-up-table (LUT) manager 435 that queries a S/D/Q LUT 440 todetermine whether the packet has been assigned a transactionidentification (TID). The S/D/Q manager 435 assigns a TID to each packetor primitive based on the S/D/Q LUT 440 query and updates the S/D/Q LUT440 in the case that a TID has not been already assigned. Althoughprimitives do not have S/D/Q information embedded in them, the networkprocessor can determine the S/D/Q that the primitive is associated withbased on the packets before or after the primitive.

The TID is received by a path manager 445, which queries a path LUT 450.The path manager 445 determines an appropriate path based on the pathLUT 450 query thereby indicating an analysis processor 455 a-n assignedto the packet's TID. As a result, all packets and primitives that belongto the same transaction can be sent to the same analysis processor 455a-n. The path manager 445 forwards the TID to a TID and timestamp (TS)interleaver 460 that interleaves the TID with a TS signal received froma TS counter 462. The interleaved TID and TS are routed to thedistribution module 465 followed by the corresponding data packet.Additional fields from the network data or primitive can be extracted bya SOF/EOF parser 470 and communicated to the path manager 445 along withother appropriate information so that the path manager 445 can establishan appropriate path for the correct duration for transfer of the networkdata. For example, the path manager 445 can receive open, close, startof frame (SOF) and end of frame (EOF) fields extracted from the networkdata by the SOF/EOF parser 470. In some instances, the path manager 445may be able to leave an established path open for transfer of more thanone piece of network data. The path manager 445 can leave an establishedpath open until a different path needs be established.

The distribution module 465 receives the interleaved TID/TS followed bythe network data from the network processor 425 and routes them to oneof several possible outputs 475 a-n according to control signalsreceived from the network processor 425. For example, the networkprocessor 425 can provide “select” and “enable” control signals forselecting one of several outputs 475 a-n of a distribution module 465and establishing a path by enabling such output to receive and transferthe network data packet to an appropriate FIFO memory buffer 480 a-n.The other outputs of the distribution module 465 can be either held inthe inactive state or open-circuited, depending on the type ofdistribution module 465.

Upon routing the network data packet to a particular output 475 a-n ofthe distribution module 465, the network data is received within one ofthe plurality of FIFO memory buffers 480 a-n. The FIFO memory bufferthat receives the primitive or data packet (e.g., FIFO 480 a) allows thereceived data to “fall through” to the FIFO's output queue with only asmall delay. Input and output from the FIFO buffers 480 a-n arecontrolled by separate clocks in one embodiment, and each FIFO 480 a-ntracks what data has entered the FIFO 480 a-n and what data has beenremoved from the FIFO's 480 a-n queue. Each FIFO 480 a-n can send astatus signal to the network processor 425 indicating an amount of datastored in the particular FIFO's queue (e.g., FIFO 480 a). The statussignal, for example, can be used for load balancing or to change how theanalysis processors 445 a-n analyze the network data. Analysis, forexample, can be prioritized, filtered, or otherwise altered using thestatus signal. The analysis performed by the processors 455 a-n can bealtered using other criteria than the status signal.

The routing of the network data to each of the FIFO memory buffers 480a-n can be controlled by the network processor 425 such that the FIFOs480 a-n are not allowed to fill up completely, and so that theprocessing of the network data received from the network 405 can bedistributed appropriately between the different analysis processors 455a-n for load balancing or for other purposes. For example, the FIFO 480a receiving the network data can next forward the network data to itscorresponding analysis processor 455 a coupled to the FIFO 480 a toanalyze the network data and store the results of the analysis (e.g.,any data including errors) in a HDD that can be internal or external tothe analysis processors 455 a-n. An additional storage processor canalso be implemented and can include ready access memory for caching andmanaging the network data storage processes.

As discussed above, several different devices can be implemented toperform the tasks and processes described herein. For example, referringto FIG. 4B, a network analyzer 402 is shown according to an exampleembodiment of the present invention. The network analyzer 402 is similarto that shown in FIG. 4A, but network analyzer 402 can include afront-end FPGA 422 for receiving the network data and performingprocesses prior to the network data being received by the networkprocessor 425. For example, the front end FPGA 422, or other logicdevice can interleave timestamp data into the network data before thenetwork data is received by the network processor 425. In addition, aback-end FPGA 432 can be included as a distribution module forperforming the demultiplexing and routing of the network data tomultiple analysis processor 455 a-n.

Another advantage of having several channels for network analysis isfault tolerancing. Fault tolerancing, as used herein, compensates forfailure of a particular channel of an analysis system. For example, inthe instance that analysis channel providing network data to theanalysis processor 455 n fails for any reason, the analysis channelproviding network data to analysis processor 455 n will still analyzethe network data and the FPGA can route the data intended for analysisprocessor 455 a to analysis processor 455 n and other analysis processorin the system. Methods of filtering network data and prioritizedanalysis can be implemented with consideration of the failed analysischannel.

The FPGA 245 can receive the network data before it is provided to thenetwork processor 425, which is one embodiment of the network processor230. The FPGA 422 can also modify the frames or packets of the networkdata stream. For example, the payload of a frame can be completely orpartially removed, and statistics can be inserted in its place. Thisway, the network data payload need not be passed to the networkprocessor in all instances. This enables the network processor to handlea data stream of a larger bandwidth than the network processor wouldtypically be able to handle. Tick frames can also be generated andinterleaved similar to that described above with reference totimestamps. Tick frames will signal the network processor that a certainamount of time (e.g., 1 second) has elapsed and will signal the networkprocessor to upload statistics to an analysis processor. Primitives canbe combined with a Timestamp into a special frame and provided to thenetwork processor.

The present invention may facilitate analysis of data in packet switchednetworks. When transferring data from a source to a destination thenetwork data is often transmitted in packets of data, each packet makingup a portion of a transaction. Each transaction can be broken intopackets of a certain size in bytes. Each packet can carry with it theinformation that will help it get to its destination and identify thepacket or the transaction to which it belongs. For example, the packetmay include the source network or IP address, the intended destination'snetwork or IP address, information that tells the network how manypackets the transaction has been broken into and identifies theparticular packet. The packets carry the network data in the protocolsthat the network uses and each packet contains part of the network datamaking up a transaction.

Depending on the type of network, packets of data and portions of thenetwork data stream can also be referred to as frame, block, cell,segment, etc. A packet can include many different fields such as forexample, a header, a body, and a footer. The packet can be parsed toaccess the desired information in each field of the packet. The packetcan contain instructions about the network data carried by the packet.These instructions may include the length of a packet (e.g., somenetworks have fixed-length packets, while others rely on the header tocontain this information), synchronization (e.g., a few bits that helpthe packet match up to the network), packet identification number or “Q”number (e.g., which packet this is in a sequence of packets), protocol(e.g., on networks that carry multiple types of information, theprotocol can define what type of packet is being transmitted (e.g.,e-mail, web page, streaming video), destination address (where thepacket is going), and originating address (where the packet came from).Generally, the body, or data payload, of a packet is the actual datathat the packet is delivering to the destination. Some networkprotocols, such as Fibre Channel, also have Primitives which typicallycarry information associated with the lower layers of the protocol. SomePrimitives carry information about the transaction they reside in. Otherprimitives may carry information that spans multiple transactions.

According to an aspect of embodiments of the present invention the frontend FPGA 422, or other logic device, can create a special header foreach packet and/or primitive. The header can contain a timestamp, and incases where multiple ports are receiving network data being analyzed cancontain a port number. Multiple ports can be defined as simply as a Hostport 410 and a Target port 415 as shown in FIG. 4B, or can be a largernumber of ports since some protocols, such as Serial Attached SCSI, usemultiple serial data streams for higher bandwidth, and the network datawithin a single transaction may be sent on any of the multiple ports.These multi-port/single-transaction streams can introduce new types ofprotocol errors associated with port selection and port management andthe analyzer can keep track of which port the packet or primitive camein on in order to troubleshoot port-related issues.

An encapsulated Packet can contain the fields such as Header Type(type=packet), Timestamp, Port Number, and the original packet, forexample. An encapsulated primitive can contain fields such as HeaderType (type=primitive), Timestamp, Port Number, Repetitive PrimitiveCount, and the original primitive. A Repetitive Primitive Count can beused if the front-end FPGA counts repetitive primitives and sends themto the network processor as a primitive value and count instead ofsending each repetitive primitive individually.

For example, with cross reference to FIGS. 4B and 4C, a typical frame480, a modified frame 486, and a modified primitive 493 are shownaccording to an example embodiment of the present invention. The typicalframe 480 can include SOF 481, header 482, payload 483, CRC 484 and EOF485 portions. The modified frame 486 can include an ID portion 487, astats portion 488, and the original frame optionally excluding thepayload 491 of the original frame. The stats portion 487 can include aport number, a timestamp, or other descriptive information. The ID 487can be an identification assigned by the FPGA 422 or other device of theanalyzer 480. The payload 491 can be excluded from the packet for anyreason. For example, the payload 491 can be excluded where analysis ofthe payload 491 will be excluded and as a result there is no need totransmit the payload 491. An indication that the payload 491 has beenexcluded can be written to the stats portion 488, or other portion ofthe modified frame 486.

The modified primitive 493 can include an ID 494, a timestamp field 495,a port number field 496, and any other information, which can be writtento the modified primitive 493 by the front end FPGA 422 or other device.Repetition count information 497 can be written to the modifiedprimitive 493 where multiple primitives have been excluded and therepetition count portion 497 of the modified primitive 493 can describethe total number of primitives excluded but described by the primitive440. The value of the primitives value 498 can be included with theprimitive 493 so that an analysis processors 455 a-n can keep track ofthe number of primitives excluded from analysis and their value.

According to embodiments of the present invention, a SPI4.2 header canbe placed on the network data by the network processor 425 or one of theFPGAs 422 or 432 for purposes of directing the network data to aspecific output port. One of the FPGAs 422 or 432 can be used for therouting. However, any router chip compliant with SPI4.2 can be used toperform the job of the distribution module discussed herein. There aremany ASICs designed as SPI4.2 routers that can do the job ofdistribution and any embodiments of the present invention can includeany SPI4.2 router.

According to another aspect, pre-analyzing by the Network Processor 425or the FPGAs 422 or 432 can be conducted so that the amount of analysisperformed by the analysis processors 455 a-n is reduced, or the amountof data that is transferred to the analysis processors 455 a-n isreduced. For example, rather than providing each packet and eachprimitive in a transaction to an analysis processor 455 a-n, the networkprocessor 425 can summarize each transaction and only provide thesummary data to the analysis processor 455 a-n. For example a summarycould include the S/D/Q, a command, a response, a number of payloadbytes, a number of payload packets, a start time, and an end time.

Referring still to FIG. 4B, the network analyzer 402 can receive networkdata from one or more physical connections 420 placed on a network linkand forward the copy of the network data to the front-end FPGA 422. Ifmultiple physical links or ports are being analyzed, the front-end FPGAcan keep track of which port the network data came from. The front-endFPGA 422 receives the network data and receives a timestamp value from atimestamp counter 462. The front-end FPGA 422 interleaves or inserts thetimestamp values with the port number and network data and forwards theresulting network data, port number, and timestamp data to the networkprocessor 425. The network processor 425 can receive the network datastream including the timestamp data. A S/D/Q parser 430 can extractidentification fields from the packets of data. The S/D/Q informationcan be sent to a S/D/Q look-up-table (LUT) manager 435 that queries aS/D/Q LUT 440 to determine whether the S/D/Q has been assigned atransaction identification (TID). The S/D/Q manager 435 can assign a TIDto each packet or primitive based on the S/D/Q LUT 440 query and updatesthe S/D/Q LUT 440 in the case that a TID had not been already assigned.

The TID can be received by a path manager 445, which queries a path LUT450. The path manager 445 can determine an appropriate path based on thepath LUT 450 query thereby indicating an analysis processor 455 a-nassigned to the packet's TID. As a result. The path manager can 445interleave or insert path and/or TID data with the network data using aTID/path interleaver 447. The timestamp, path, and/or TID data can beinterleaved with the network data as fields included with each packet ofdata or primitive, or added as an additional header to each packet orprimtive. The interleaved path, TID, timestamp, and network data can betransmitted to the back-end FPGA 432.

The back-end FPGA 432 can receive the network data and route it to oneof several possible outputs according to the path or TID datainterleaved with the network data. The path data, as well as additionalfields from the network data packet or primitive can be extracted by theback-end FPGA 432 along with other appropriate information so that theFPGA 432 can establish an appropriate path for the correct duration totransfer the primitive or packet of data to one of the FIFO memorybuffers 480 a-n. The back-end FPGA 432 and a demultiplexer are examplesof distribution modules.

Upon routing the primitive or data packet to a particular output of theback-end FPGA 432, the network data can be received within one of theplurality of FIFO memory buffers 480 a-n. Each FIFO 480 a-n can send astatus signal to the network processor 425 indicating an amount of datastored in the particular FIFO's queue (e.g., FIFO 480 a). The routing ofthe network data to each of the FIFO memory buffers 480 a-n can becontrolled by the network processor 425 and carried out by the back-endFPGA 432 such that the FIFO memory buffers 480 a-n are not allowed tofill up completely, and so that the processing of the network datareceived from the network 405 can be distributed appropriately betweenthe different analysis processors 455 a-n.

The FIFO 480 a-n receiving the network data next forwards the networkdata to its corresponding analysis processor 455 a-n to analyze thenetwork data and store the results of the analysis in a HDD inside theanalysis processor 455 a-n. An additional storage processor can also beimplemented and can include ready access memory for caching and managingthe network data storage processes. Also, multiple storage mediums, suchas HDDs, can be coupled to each analysis processor 455 a-n for storingnetwork data for later retrieval and analysis as needed.

Referring to FIG. 5, a storage processor 500 is shown in conjunctionwith a FIFO memory buffer 505, local memory 520, and an analysisprocessor 510 illustrating an example of how additional processors andhardware can be implemented according to the present invention. The FIFOmemory buffer 505 can receive the network data from a distributionmodule, such as a demultiplexer or a FPGA (see e.g., FIGS. 4A and 4B),which can be preceded by an interleaved TID and TS. The FIFO 505 canalso send a FIFO status to be received by a network processor forcontrolling and directing data to an appropriate FIFO buffer andanalysis processor.

The FIFO memory buffer 505 forwards the network data to the storageprocessor 500 that is coupled to a HDD 515 and can work in conjunctionwith the analysis processor 510 for storage of data in the HDD 515including storage of errors, network data, and storage of results of ananalysis conducted by the analysis processor 510. The storage processor500 can also store network data that has not been fully analyzed (e.g.,because it has been selected for filtering or only partially analyzed asdiscussed in further detail herein) and can be later retrieved andforwarded to the analysis processor 510 for processing. The storageprocessor 500 can be any type of appropriate processor.

It should be appreciated that many of the embodiments of the presentinvention can be carried out using a single processor coupled to a harddisk drive and local memory doing the entire job of analyzing data fromthe FIFO without the need for additional storage or an additionalstorage processor. Moreover, many embodiments of the present inventioncan be carried out using only a computer, which can be coupled to a FIFOmemory buffer receiving network data, and additional components may notbe required.

2. Demultiplexing with Distributed Analysis

In one embodiment, only one analysis processor receives a piece ofnetwork data. According to another embodiment of the present invention,the same network data can be sent to multiple analysis processors usinga data distribution module such as a demultiplexer or an FPGA. Thisgives a network analyzer the capability of sending a single input datastream to any number of outputs of the distribution module includingmultiple outputs of the distribution module. Each output of thedistribution module can be coupled to a different analysis processor andany number of the coupled analysis processors can potentially analyzethe same data for any number of analysis tests or layers of analysis.The routing of the network data to the analysis processors, as well asthe type of analysis conducted on the network data at each processor,can be determined on any basis. For example, the routing of the networkdata to the analysis processors, and the tests conducted on the networkdata at each analysis processor can be determined based at least in parton the amount of data stored in a memory buffer coupled to an analysisprocessor.

Referring now to FIG. 6, a system 600 for analyzing data is shownaccording to an example embodiment of the present invention. A networklink 602 transmitting a data stream between a source 605 and adestination 610 in a network 615 can be tapped and network datarepresenting at least a portion of the network data stream can bereceived by a network processor 620. The network data can be transmittedto a distribution module 625 along with a control signal from thenetwork processor 620. The distribution module 625 can be ademultiplexer capable of routing the network data to multiple outputs ofthe distribution module 625 based on the control signal received fromthe network processor 620. Front-end and/or back-end FPGAs can also beimplemented as discussed above with reference to FIG. 4B in this or anyof the embodiments described herein.

Each output from the distribution module 625 can be coupled to a memorybuffer 630 a-n (e.g., a FIFO memory buffer). Each memory buffer 630 a-nthat receives the network data acts as a data buffer and provides thenetwork data in turn to a corresponding analysis processor 635 a-n. Anynumber of the analysis processors 635 a-n may be configured to conductdifferent analysis tests on the network data received than otheranalysis processors. The analysis tests conducted by any of the analysisprocessors 635 a-n can be determined based on any appropriate basis. Forexample, the analysis tests of at least two of the analysis processors635 a-n receiving the same data can be different, thereby distributingthe processing burden of a single piece of network data across multipleanalysis processors 635 a-n. The analysis processors 635 a-n can be incommunication with other analysis processors 635 a-n and/or the networkprocessor 620 to dynamically coordinate the testing of data, and/or tomonitor the amount of data in the memory buffers 630 a-n.

For example, analysis processor 635 a can perform analysis that verifiesthe structure of headers in the network data. Analysis processor 635 bcan perform analysis that verifies content, rather than structure of theheaders, such as values within the fields of the same network data.Analysis processor 635 c can perform analysis that verifies the protocolpayload of the same network data. Analysis processor 635 n can performanalysis that verifies the primitive handshakes and/or initializationsequence of the same network data. In this manner, the processing burdenfor these various analyses and tests can be distributed between thevarious analysis processors 635 a-n. Further, the number of tests can beincreased since the bandwidth of processing power has been increased bysuch a system.

According to an example embodiment of the present invention, theanalysis tests and routing of data can be dynamically determined based,in one embodiment, on the amount of data stored in a memory buffer. Forexample, where the amount of data stored in a memory buffer attached toan analysis processor reaches a predetermined amount, incoming data canalso be routed to additional analysis processors and the number and/ortypes of tests conducted by the processors can be distributed betweenthe processors. The analysis and tests can be distributed between theprocessors to distribute the processing burden, and/or the same test canbe run by multiple processors where redundant testing is desirable forexample. In one embodiment, the analysis performed at the analysisprocessors 635 a-n can be adjusted dynamically. The distribution of datato the various analysis processors as well as the specific analysisperformed at those analysis processors can be adjusted on-the-fly andcan be based, by way of example, on current network conditions, FIFOstatus, the need to perform specific tests, and the like, or anycombination thereof. In an embodiment where multiple processors areanalyzing the same network data, it can be desirable for a communicationpath to exist between the processors so that they may coordinate anefficient means of dividing the workload of analysis processing (e.g.,load balancing). Inter-processor communication channels are well knownin the art.

Many different methods for practicing embodiments of the presentinvention can be implemented. For example, referring to FIG. 7 a blockdiagram illustration of a method for analyzing data is shown where thesame data is distributed to other analysis processors such thatadditional or different tests can be performed on the same data. Networkdata is received (700) representing at least a portion of a data stream.A status signal can be received (710) from a FIFO buffer or from ananalysis processor and compared to a threshold (720) although the statussignal may reflect the threshold of the buffer. In other words, thestatus signal from a particular buffer can represent an amount of datain that buffer. The threshold can represent an amount of data in amemory buffer at which point analysis processing of the same networkdata will be distributed across multiple processors or at which pointthe network data is directed to other analysis processors to achieveload balancing.

In the instance that the status signal indicates that the threshold ofthe buffer is not yet reached, the network data may be forwarded to ananalysis processor corresponding with that buffer for network analysis(750). In the instance that the status signal indicates that thethreshold has been passed (or a condition has been met), the networkdata can be routed to additional analysis processors (730) and thenetwork analysis tests can be distributed between the analysisprocessors receiving the network data (740). The network analysis isconducted on the network data by the appropriate analysis processors(750). Although this example illustrates that the same network data isdistributed to other analysis processors based on the status of thebuffer, the same network data can be distributed to other analysisprocessors for other reasons as well. For example, it may be the casethat each analysis processor can more efficiently implement a particularset of tests for a given set of network data and network data can bedistributed based on this condition.

3. Filtering Data

According to another aspect of the present invention, at least a portionof a network data stream such as data packets, primitives, ortransactions can be selectively filtered such that they are selectivelyexcluded from further analysis. Network data can be excluded for anyappropriate reason. For example repetitive network data, or network datawhere the result of analysis of the network data is known can beexcluded. This may be advantageous where large amounts of repetitivenetwork data transactions will require several layers of expert analysisand produce an undesirable burden on an analysis apparatus. Thefiltering can also be based on various attributes of the network datatransfer mechanisms, protocols, and transactions.

According to example embodiments of the present invention, a filter LUTcan be maintained to identify network data and track the results ofdifferent analysis processes conducted during a predefined time frame.The filter LUT can be organized based on transactions such as aparticular source and destination pair. The filter LUT can further keeptrack of whether any errors were discovered by particular testsconducted on data transferred between a source and destination pair.

In this manner, the processing bandwidth can be reserved for morecritical analysis such as those analyses that have not been conducted,more critical network data, or that have a history of finding errors.However, it can also be set up so that the filter LUT is at leastpartially cleared after a period of time. For example, at certainintervals at least a portion of the filter LUT can be cleared such thata fresh history of transactions and errors are recorded periodically.The time period can also be based in part on the frequency of errorsdiscovered, how critical detection of the errors is, or other attributesof the network data or network analysis.

Referring now to FIG. 8, a flow diagram illustrating a method foranalyzing a network data stream implementing filtering techniques isdepicted according to an example embodiment of the present invention. Adata stream is received from a network (800). It can be determinedwhether filtering techniques should be implemented (815). For example, athreshold (or other condition) can be compared to a status signal from amemory buffer. The threshold can be a predetermined threshold amount ofdata in a memory buffer at which point certain data, such as repetitivedata or data with a known analysis result, will be excluded from furtheranalysis. In the instance that filtering is not proper, the packet isnot selected for filtering. If filtering is not implemented, the packetcan be routed to an appropriate memory buffer and subsequent expertanalysis can be conducted (820). The results of the analysis can bestored in a HDD or presented to a user via a display or printer (825).

In the instance that filtering is proper, for example when the amount ofdata stored in the memory buffer is larger than the threshold or acondition is met, the packet is selected for filtering and it isdetermined whether the network data is of the type designated forexclusion from analysis (830). The threshold can be any amount orcondition and the comparison can be conducted at predeterminedintervals. For example, the threshold can be equal to a status signalindicating that a memory buffer is at least 70% full. The threshold canrepresent the queue level in a single memory buffer, or can bedetermined from a combination of any number of memory buffers. In theinstance that the network data or transaction is of the type indicatedfor exclusion, the network data can be excluded from analysis and anindication of such exclusion can be saved to a memory or presented to auser (825). In the instance that the network data or transaction is notthe type indicated for exclusion, the network data can be forwarded toan appropriate memory buffer for subsequent expert analysis processing(820).

The network data can be excluded, for example, by not establishing alink to an analysis processor for processing the network data. Anindication of the network data exclusion can be accomplished, forexample, by establishing a communication link to an appropriateprocessor for only a TID/TS signal without a network data payloadindicating that the signal was excluded from further analysis.

Several different levels of filtering can also be implemented for statussignals indicating different amounts of network data in a memory bufferqueue. For example, if a memory buffer is 70% full a first level of datafiltering may be selected such that a first group of network data isselectively excluded from further analysis processing. Additionally, asecond higher level of filtering can be selected when the memory bufferreaches 85% full selecting a second additional group of network data forexclusion from analysis processing. Any number of levels of filteringcan be implemented, and groups of network data including contents offiltering LUTs designated for filtering can be defined by anyappropriate means. Filtering can also be implemented by altering theanalysis performed by the analysis processors. For example, the numberof tests performed by the analysis processors can be reduced. As thestatus signal from the buffer changes, the tests performed by theanalysis processors can be adapted accordingly in an example embodiment.

Apparatuses for practicing a method of analyzing a data streamimplementing filtering techniques can be embodied in a number ofconfigurations, combinations of mechanisms, and sequence of processes.For example, referring now to FIG. 9, a network analysis system 900implementing filtering techniques, such as those described above, isshown according to an example embodiment of the present invention. Anetwork data stream is received from a bidirectional network link 905between a host 910 and a target 915 representing a portion of acommunication network 920. Network data representing at least a portionof the network data stream is transmitted to a network processor 925.The network processor 925 can include a S/D/Q parser 930 that extractsS/D/Q fields from the network data and forwards the S/D/Q information toa S/D/Q LUT manager 935. The S/D/Q LUT manager 935 can access a SDQ LUT940 and assign the network data a TID that is sent to a filter manager945. The filter manager 945 can receive a status signal from a FIFOmemory buffer 950 coupled to an analysis processor 955. The statussignal may indicate the amount of data stored in the FIFO memory buffer950 for example. The filter manager 945 can compare the status signal toa filter threshold to determine whether to begin excluding data fromfurther processing by the analysis processor 955.

In the instance that the status signal is smaller than the thresholdlevel (e.g., indicating that the amount of data stored in the FIFO's 950queue is lower than a threshold amount), filtering is not selected andthe filter manager 945 can communicate with a path control parser 960 soas to forward the network data packet to the FIFO memory buffer 950 forsubsequent processing by the analysis processor 955. Results of theanalysis can be saved to memory or presented to a user.

In the instance that the status signal is greater than the threshold(e.g., indicating that the amount of data stored in the FIFO's 950 queueis above a threshold amount) filtering can be selected and the filtermanager 945 can access a filter manager LUT 965 to determine whether thenetwork data is of a type selected for exclusion. In the instance thatthe network data is of the type selected for exclusion (e.g., thenetwork data is repetitive or analysis results are known), the networkdata is excluded from analysis for errors by the analysis processor 955.In the instance that the network data is not of the type indicated forexclusion from analysis, the network data can be forwarded to the FIFOmemory buffer 950 for analysis by the corresponding analysis processor955. Results of the analysis, or an indication that the network data wasexcluded from the analysis, can be saved to memory or presented to auser.

It should be appreciated that FIG. 9 depicts an example embodimentincluding a single FIFO 950 and analysis processor 955 withoutadditional demultiplexing aspects of the present invention discussedherein. According to other embodiments of the present invention, thefiltering techniques can be combined with other aspects of the presentinvention. Data and transactions can be filtered and demultiplexed toone of several memory buffers and several corresponding analysisprocessors. In addition, the analysis processors can also receive thestatus signal as well as communicate with the network processor in orderto implement filtering at the analysis processors such that the numberof tests, or network data tested, is adjusted.

For example, referring to FIG. 10, a block diagram illustrating a methodfor analyzing data including filtering and demultiplexing techniques isshown according to an example embodiment of the present invention. Anetwork data stream is received (1000) and a network data can beidentified (1005). Transaction identification can be assigned to thenetwork data (1010) and it can be determined whether the identificationhas been assigned to an analysis processor (1015). In the instance thatthe identification has not been assigned to an analysis processor,signals can be compared (1020) from all memory buffers connected to adistribution module and the TID can be assigned to an appropriate memorybuffer (1025) (e.g., a memory buffer with the least amount of networkdata stored therein). It should be understood that the identificationcan be assigned on any basis. For example, the identification can beassigned to a particular memory buffer and analysis processor based onthe particular transaction, source, destination, data type, protocol,etc.

After an appropriate memory buffer and analysis processor have beenassigned, a signal from the assigned memory buffer can be compared to afilter threshold (1035). The filter threshold can be an amount ofnetwork data in the memory buffer's queue (e.g., at least about 70%-85%of capacity) at which point filtering techniques will be implemented. Inthe instance that the signal indicating the amount of data stored in thememory buffer is less than the filter threshold, the network data can besent to the assigned memory buffer and expert analysis can be conductedby the assigned analysis processor (1040). Results of the analysis ornetwork data including errors can be stored and/or presented to a user(1045).

In the instance that the signal from the memory buffer is greater thanthe filter threshold, it can next be determined whether the network datais of the type indicated for exclusion from expert analysis (1050). Ifthe network data is not of the type indicated for exclusion, the networkdata can be forwarded to the appropriate memory buffer and analysisprocessor, and expert analysis can be conducted on the network data(1040). A result of the analysis can be stored and/or displayed (1045).In the instance that the network data packet is of the type indicatedfor exclusion from analysis, the network data may not be analyzed(1055), but the TID and an indication that the network data was notanalyzed can be stored in memory and/or presented to a user 1045.

It should be appreciated that different levels of filtering can beimplemented for different amounts of data in the memory buffer queues.For example, there can be two or more filtering thresholds thatcorrespond to different levels of filtering at different amounts ofnetwork data in the applicable FIFO queues. Different types of filteringcan also be implemented where the network data is assigned to adifferent analysis processor for different analysis processing tests(e.g., less testing) based on a status signal received from a memorybuffer. Also, filtering can be implemented independently of the statusof the memory buffer queues. Filtering can implemented based oncharacteristics of the network data itself as previously described, onspecific needs of a network operator, and the like.

4. Prioritized Analysis

According to another aspect of the present invention, analysisalgorithms and tests can be prioritized and selectively conducted on thenetwork data. The priority of each test can be selected on any basis.The priority of analysis can be selected by a user and/or can bedynamically selected by an apparatus such as embedded code in aprocessor or computer instructions loaded onto a processor. For example,tests may be prioritized based on at least one of whether the test hasbeen run on a particular data type or transaction, whether the test hasbeen conducted during a predetermined time period, the layer in whichthe test analyzes (e.g., refer to FIG. 1), the likelihood of findingerrors, and whether the test is lower in network system priority (e.g.,performance tests may be lower in priority than basic functionalitytests).

For example, referring now to FIG. 11, a block diagram illustrating amethod of performing priority analysis on network data is shownaccording to an example embodiment of the present invention. Networkdata from a transmitted data stream, or a copy representing network datafrom the network data stream, can be received (1100). Identification canbe assigned to the network data (1110). It is determined whetherpriority analysis is proper and should be implemented (1120). Priorityanalysis can be proper when, for example, a status signal from a memorybuffer indicates an amount of data stored in the memory buffer is abovea priority threshold, or priority analysis has been selected by a user.

In the instance that priority analysis is not proper, the network datacan be forwarded to an appropriate analysis processor for analysis(1140). Analysis can be conducted on the network data (1140) and aresult of the analysis can be stored in memory or presented to a user(1150). In the instance that the signal from the memory buffer isproper, the analysis can be prioritized (1130) and the prioritizedanalysis can be conducted (1140) on the network data. The network data,a description of any prioritization of tests, and/or a result of theanalysis can be stored in memory or presented to a user (1150).

Prioritization of the different tests and analysis algorithms can bebased on a variety of factors. For example, priority may be based on atleast one of whether the test has been run on a particular network datatype or transaction in a given time frame and whether the test is lowerin system priority, for example.

Memory in a processor can be compiled to keep track of informationrelated to processes conducted and the memory can be queried and updatedusing any appropriate means (e.g., an analysis processor or a networkprocessor) in an analysis system implementing the methods of the presentinvention. Moreover, different analysis processors in a demultiplexedsystem can prioritize tests differently and maintain separate priorityLUTs. Different tests can also be prioritized differently for differenttransactions, protocols, mechanisms, and network conditions.

Apparatuses for practicing the methods for prioritizing and analyzingdata of the present invention can be embodied in various configurationsand process sequences. For example, referring to FIG. 12, a system 1200for prioritizing and analyzing data received from a network 1210 isshown according to an example embodiment of the present invention. Amemory buffer 1220 receives a data stream, or a copy of the network datastream, transferred from a source 1230 to a destination 1240 in thenetwork 1210. The memory buffer 1220 forwards the network data in turnto an analysis processor 1250. A memory buffer status signal can bereceived by the analysis processor 1250 indicating an amount of networkdata stored in the memory buffer 1220. The analysis processor 1250 cancompare the signal to a priority threshold representing, for example, anamount of data at which point prioritization of tests will beimplemented.

The analysis processor 1250 can include a priority LUT stored in memoryand in the instance that the signal from the memory buffer 1220 isgreater than the priority threshold, the priority LUT can be queried todetermine an amount of prioritization of analysis that should beconducted for the particular transaction. The analysis processor 1250can conduct the appropriate tests for errors and store a result of thetests in memory and/or present results of the tests along with anindication of any tests not conducted due to prioritization of the teststo a user.

Referring to FIG. 13, an illustration of an example priority LUT 1300 isshown listing examples of tests and analysis algorithms that can beconducted on a transaction or piece of network data. The priority LUT1300 can be specific to a particular transaction, piece of network data,or analysis processor, or can be a general priority LUT 1300 to bequeried for every transaction or piece of network data. The priority LUT1300 can include additional associated data structures indicatinghistorical outcome of each test. The priority LUT 1300 can be a statictable, developed by user input, or a dynamically generated table updatedand maintained by the analysis system itself.

As illustrated, the priority LUT 1300 can include historical data ofwhether each test has been passed, failed, or not observed. Thus, thetests can be prioritized, for example, such that tests that have notbeen observed and tests that have historically failed are prioritizedabove tests that historically have been completed and have not founderrors in the network data tested. Like the filter LUT, the priority LUT1300 can be cleared at least in part at any interval (e.g., each day) sothat the historical outcome of every test will be determined at least atsome determined interval.

In addition, the priority LUT 1300 can include a prioritization ofdifferent tests based on the layer of analysis or how critical detectionof errors is to the operation of the network. The priority LUT 1300 canalso include multiple priority LUTs for different sets of tests thatwill be excluded. Different levels of priority analysis can beimplemented depending on the amount of data in a single memory buffer,or the amount of data stored in multiple memory buffers.

According to other example embodiments of the present invention theabove described prioritization of tests can be combined with otheraspects of the present invention discussed herein (e.g., using system1200 in FIG. 12 in some instances). For example, priority analysis canbe combined with filtering techniques and/or embodiments includingdemultiplexing of network data to multiple analysis processors.

According to an example embodiment of the present invention a networkmethod and apparatus for practicing such methods can include filteringtechniques, prioritized analysis techniques, and demultiplexing of datato multiple analysis processors, which are aspects of severalembodiments of the present invention discussed herein. For example,referring to FIG. 14, a block diagram illustrating a method foranalyzing network data is shown according to an example embodiment ofthe present invention. A network data stream, or a copy of the networkdata stream, can be received (1400). Network data relating to aparticular transaction can be separated and the identificationinformation can be extracted (1405). The network data can be assigned anidentification 1410 and it can be determined whether the identificationhas been assigned to an analysis processor (1415). The network data canbe prepared and additional fields can be used. Also, portions of thenetwork data, such as a payload or fields can be excluded.

In the instance that the identification has not been assigned to ananalysis processor, the identification can be assigned to an analysisprocessor (1420). It should be understood that any criteria can be usedto assign the TID to an analysis processor such as, for example,assigning the TID to the analysis processor coupled to a memory bufferwith the lowest amount of data in its queue, assigned based on the typeof transaction the network data belongs to, or assigned based on thetype of analysis conducted by the analysis processor.

After the appropriate analysis processor has been assigned, a statussignal indicating an amount of data stored in a memory buffer coupled tothe assigned analysis processor can be compared to a filter threshold(1425). The status signal, as previously stated, may be a binary flagindicating whether the buffer can receive additional data. The filterthreshold can equal an amount of data stored in a memory buffer at whichpoint the analysis system will start to remove certain packets ortransactions of data from analysis processing. In the instance that theamount of data stored by the memory buffer is above the filteringthreshold, it can be next determined whether the network data is of thetype selected for exclusion from analysis (1430). In the instance thatthe network data is of the type for exclusion the network data can beexcluded from analysis processing, and the network data, or anindication that the network data was excluded from analysis, can besaved to memory or presented to a user (1440). In the instance that thenetwork data is saved to memory, the network data can also be laterretrieved for subsequent analysis.

In the instance that the status signal indicating amount of data in theFIFO is not above the filtering threshold or the network data is not ofthe type of data selected for exclusion, the amount of data stored inthe memory buffer can be compared to a priority threshold (1450). Thepriority threshold can be an amount of data stored in the memory bufferat which point the analysis will be conducted on data according to itspriority relative to other tests. It should be appreciated that thepriority threshold can be checked prior to the filtering threshold orthe thresholds can be staggered so that a lower threshold is comparedprior to a larger threshold requiring that only one threshold to bequeried in the instance that the status of the memory buffer is lowerthan the first threshold. Multiple levels of prioritization andfiltering can also be implemented.

In the instance that the status signal indicating an amount of datastored in the memory buffer is less than the priority threshold, thenetwork data packet can be analyzed by the assigned analysis processor(1455). In the instance that the status signal is greater than thethreshold, for example indicating that an amount of data stored in theassigned memory buffer is greater than the priority threshold, thenumber of tests conducted, layers of analysis, or level of analysis canbe prioritized (1460) and analysis can be conducted at this according tothe prioritization of analysis (1455). The results of the analysis canthen be saved to memory or presented to a user (1465).

Example embodiments of network analysis apparatuses implementingfiltering techniques and/or prioritized analysis, and/or demultiplexingand redirection of data to multiple analysis processors can be embodiedin various configurations and sequences of mechanisms for conducting thedifferent processes. For example, referring to FIG. 15, an illustrationof a system 1500 for analyzing network data including a transactiondistribution module 1505 implementing filtering and priority analysistechniques is shown according to an example embodiment of the presentinvention. The system for analyzing network data 1500 can include anetwork processor 1510 coupled to the transaction distribution module1505 where each output of the distribution module 1505 can be coupled toa different FIFO memory buffer 1515 a-n, analysis processor 1570 a-n,and storage hard disk drive 1575 a-n.

In operation, a packet of data can be received from a bidirectionalnetwork data stream by two physical connections 1520 coupled to thenetwork processor 1510. The network processor 1510 can include logic forperforming each of the described functions. The network processor 1510can include a S/D/Q parser 1575 that receives network data and extractsS/D/Q information from fields of the packet. The S/D/Q information canbe communicated to a S/D/Q LUT manager 1530. The S/D/Q LUT manager 1530can query a S/D/Q LUT 1535 and assign a TID to the network data based onthe results of the S/D/Q LUT query. The S/D/Q manager 1530 cancommunicate the TID to a filter and path manager 1540. The functions ofthe filter and path manager 1540 can be accomplished by a singleprocessor as shown, or can be accomplished by multiple processors orlogic devices including executable logic for carrying out the describedfunctions. The filter manager and path manager 1540 functionality canalso be programmed into the network processor 1510. The filter and pathmanager 1540 can receive signals indicating the status of at least oneof the FIFO memory buffers 1515 a-n coupled to corresponding analysisprocessors 1570 a-n. The filter and path manager 1540 can compare theFIFO status signal to a stored or received filter threshold to determinewhether to implement filtering techniques. In the instance that thatstatus signal is greater than the threshold, the filter and path manager1540 can compare the network data or TID to information stored in afilter manager LUT 1550 to determine whether the packet or transactionis of the type selected for exclusion from analysis. The filter and pathmanager 1540 can implement packet filtering or transaction filteringdepending on the FIFO status signal or on any other basis. The filtermanager 1540 can exclude repetitive packets or repetitive transactions,or filter by S/D/Q identification. It will not, however, filter responsepackets with a bad error status in the filter LUT 1550.

The filter and path manager 1540 can also receive the TID from the S/D/QLUT manager 1530 and query a path manager LUT 1545 to determine whetherthe TID has been assigned to a particular path of the distributionmodule 1505. The path manager 1540 can ensure that all packets andprimitives which belong to the same transaction are sent to the sameanalysis processor (at least one of 1570 a-n) connected to an outputpath of the distribution module 1505. The path manager 1540 can transmitcontrol signals, such as enable and select signals, coordinated withsignals received from a SOF/EOF parser 1555 to control the path to whichthe network data is routed and the duration for which the path isestablished. The TID can be routed to a TID and TS interleaver 1560,which receives a TS signal from a TS counter 1565. The interleaver 1560can route the interleaved TS and TID to an input of the transactiondistribution module 1505 followed by the network data packet from theSOF/EOF parser 1555. Each channel of the distribution module 1505 canreceive a control signal allowing for transfer of the packet of data tothe particular processor (at least one of 1570 a-n) assigned to thetransaction.

An analysis processor 1570 a-n can also receive a status signal from itscorresponding FIFO memory buffer 1515 a-n indicating, for example, theamount of data stored in the FIFO's queue. Based on the status signalreceived from the FIFO memory buffer 1515 a-n, the analysis processor1570 a-n can query a priority LUT and prioritize the number of tests,algorithms, and/or the layers of analysis conducted on the network data.For example, expert analysis software can use at least one LUT toprioritize tests that are not observed yet, or are not as critical tothe operation of the network. If the FIFO 1515 a-n is reaching itscapacity, the analysis processor 1570 a-n can implement priorityanalysis so that testing is intelligently prioritized. A differentpriority LUT can be maintained for each source and destination pair.

The analysis processor 1570 a-n can also provide the user with constantstatus regarding the FIFO 1515 a-n fullness as well as filtering methodsused and prioritization of tests being conducted. If the user wants lessfiltering, he can reduce the amount of processing (e.g., less expertanalysis), add more processors (e.g., more demultiplexing), or use morepowerful processors. Similarly, the analysis processors 1570 a-n and thenetwork processor 1510 can communicate with each other such that if theuser wants more processing (e.g., more expert analysis and lessprioritizing of tests), the user can increase the amount of filtering,add more processors, or use more powerful processors.

The analysis processors 1575 a-n can be coupled to HDDs 1575 a-n forstorage of network data associated with transactions that have errors,protocol violations, or other anomalies. An IT manager can furtheranalyze the details of these transactions days after they occur. Theanalysis software can prioritize tests so that all tests are eventuallyrun on all source and destination pairs, but some tests can be run lessfrequently than other tests as desired. The analysis processors 1570 a-ncan store the results of the analysis conducted in the HDDs 1575 a-nand/or output the results of the analysis to a display or printer, forexample.

5. Example Embodiments Scaling the Present Invention

The present invention can also be scaled in several different aspects soas to remove bottle necks from the network data analysis system. Forexample, the present invention can be scaled at the input level, thenetwork processing level, and the distribution module level. Scaling atthe network analysis level by adding analysis processors is discussedabove.

Another advantage of scaling various embodiments of the presentinvention is for fault tolerancing. For example, where a network dataanalysis system includes multiple inputs, network processors,distribution modules, and/or multiple analysis processors allowing forseveral channels for network data, the network data analysis system canredistribute the routing and processing burden between any of thesecomponents in the case of failure of any of the components. Any of thecomponents of the network analysis system can be in communication todetect failure of a component of the system and dynamically adjustrouting of network data to insure that the network data is received byan analysis processor or storage medium and properly analyzed.

The present invention can be scaled at the input level by providingmultiple input channels or ports to the network processing system. Anetwork can be accessed at multiple links, and network data representingmultiple data streams transmitted across the network can be received bythe network analysis system simultaneously. For example, referring toFIG. 16, a network processing system 1600 is shown according to anexample embodiment of the present invention. The network processingsystem 1600 can include a first input 1605 to the network analysissystem 1600 that receives network data from a first communication link1615 between a first source 1625 and a first destination 1615.Similarly, a second input 1610 to the network analysis system 1600receives network data from a second communication link 1620 between asecond source 1635 and a second destination 1640. It should beappreciated that the same network link can also be accessed in twolocations and the network data can represent at least a portion of thesame or different data streams.

The network data can be received by a network processor 1645 thatprovides network data and a control signal to a distribution module1650. The distribution module 1650 can receive the network data andcontrol signal from the network processor 1645 and route the networkdata to at least one memory buffer 1655 a-n coupled to an analysisprocessor 1670 a-n. At least one memory buffer 1655 a-n receives thenetwork data and provides the network data in turn to its correspondinganalysis processor 1670 a-n. Each input 1605 and 1610 can also bereferred to as ports.

Referring now to FIG. 17, an example embodiment of the present inventionis shown where a network analysis system 1700 includes multiple inputsto a network processor 1702 for receiving network data. A first input1705 to the network analysis system can include two physical connections1710 to a data transmission link configured to tap and receive networkdata representing at least a portion of a data stream transmitted acrossthe network data transmission link. The network data can be received bya first S/D/Q 1715 parser that can extract S/D/Q fields from the networkdata identifying the network data by, for example, transaction, source,destination, type of data, or other appropriate identification. TheS/D/Q fields from the first S/D/Q parser 1715 can be received by S/D/QLUT manager 1720 that queries a S/D/Q LUT 1725 and assigns a TID to thenetwork data.

Similarly, a second input 1735 to the network analysis system 1700 caninclude two physical connections 1730 to another data transmission link.The second input 1735 can be configured to receive network datarepresenting at least a portion of a data stream transmitted across thesecond data transmission link. The network data can be received by asecond S/D/Q parser 1740 that extracts S/D/Q fields from the networkdata identifying the network data. The S/D/Q fields from the secondS/D/Q parser 1740 can be received by the S/D/Q LUT manager 1720 that canquery the S/D/Q LUT 1725 and assign a TID to the network data. The TIDcan be received by a path manager 1750 that queries a path LUT 1755 andcommunicates with a path control field parser 1760 and a distributionmodule 1765 to route the network data received by both inputs to anappropriate memory buffer 1770 a-n coupled to a corresponding analysisprocessor 1775 a-n. A serializer-deserializer can also be used toserialize data received from multiple connections in a single datastream input to the analysis system 1700.

Referring now to FIG. 18, an example embodiment of the present inventionis shown where multiple network processors 1800 a-n have beenimplemented. The network processors 1800 a-n can be part of the samenetwork analysis system 1805, each network processor 1800 a-n receivingthe same or different network data.

For example, a first network processor 1800 a can receive network datarepresenting at least a portion of a data stream transmitted between afirst source 1810 and a first destination 1815 in a network 1820. Thenetwork data can be received by a memory buffer 1825 a from the firstnetwork processor 1800 a and the memory buffer 1825 a can provide thenetwork data in turn to a corresponding analysis processor 1830 a.Similarly, a second network processor 1800 n can receive network datarepresenting at least a portion of a data stream transmitted between asecond source 1835 and a second destination 1840 in the network 1820 ordifferent networks. The network data can be received by a second memorybuffer 1825 n and the network data can be provided in turn to a secondanalysis processor 1830 n for analysis of the network data. The firstnetwork processor 1800 a can be coupled to the second network processor1800 n so that network data, transaction data, control data, memorybuffer status data, and/or analysis data can be shared between thenetwork processors 1800 a-n.

Referring to FIG. 19, an example embodiment of the present invention isshown implementing multiple network processors 1900 a-n and multiplenetwork data distribution modules 1905 a-n (e.g., FPGAs). A firstnetwork processor 1900 a receives network data representing at least aportion of a data stream transmitted between a first source 1910 and afirst destination 1915 in a first network 1920. The network data can bereceived from the network processor 1900 a along with a control signalby a first distribution module 1905 a. The first distribution module1905 a can route the network data to at least one of its outputs coupledto a corresponding FIFO memory buffer 1925 a-b and analysis processor1930 a-b.

Similarly, the second network processor 1900 n receives network datarepresenting at least a portion of a data stream transmitted between asecond source 1935 and a second destination 1940 in a second network1945. The network data can be received by a second network datadistribution module 1905 n from the second network processor 1900 nalong with a control signal. The second distribution module 1905 n canroute the network data to at least one of its outputs that is coupled toa corresponding FIFO memory buffer 1925 c-n and analysis processor 1930c-n.

Any of the network processors and analysis processors can be coupled forcommunication to another network processor (or logic device) to sharecontrol data and/or network data. For example, the first and secondnetwork processors 1905 a and 1905 n can share information related totransactions, LUTs, network errors, distribution module control data,memory buffer status data, and analysis control data. Each distributionmodule 1905 a-n can also include a connection from at least one output(e.g., 1945 a-n) to the input (e.g., 1950 a-n) of another distributionmodule 1905 a-n. For example, as shown in FIG. 19, an output 1945 a ofthe first distribution module 1905 n can be coupled to an input 1950 nof the second distribution module 1905 n. Similarly, an output 1945 n ofthe second distribution module 1905 n can be coupled to an input 1950 aof the first distribution module 1905 a. Accordingly, each networkprocessor 1900 a-n can transmit data to any of the analysis processors1900 a-n coupled to either distribution module 1905 a-n by coordinatingcontrol information with the other network processor 1900 a-n toestablish an appropriate path of both distribution modules 1905 a-n.

Allowing transfer of network data as well as control information betweenthe network processors 1900 a-n and distribution module channels can beadvantageous for many reasons. For example, this embodiment may allowfor sampling data as it passes through different channels and protocols.When analyzing data at the network layer it may be advantageous toanalyze data both prior to a protocol conversion and following aprotocol conversion. In this manner, the first data stream 1605 (e.g., aFibre Channel data stream) can be received (e.g., by network processor1900 a) prior to the network data stream 1605 undergoing a protocolconversion (e.g., Fibre Channel to Ethernet). The second data stream1610 (e.g., an Ethernet data stream) can be received (e.g., by networkprocessor 1900 n) following the second data stream 1610 undergoing theprotocol conversion (e.g., Fibre Channel to Ethernet). According to thisembodiment of the present invention the first data stream 1605 can bedirected to the same analysis processor as the second data stream 1610by directing one of the network data streams to the other distributionmodule (e.g., using output 1945 a to direct the first stream 1605 fromdistribution module 1905 a to input 1950 n of distribution module 1905n). Thus, the network data may have originated in the same form, but a“before and after” depiction can be received by any of the analysisprocessors 1930 a-n coupled to either network processor 1900 a-n by thedistribution modules 1905 a-n. Each network processor 1900 a-n can alsoreceive a different type of signal from a different type of link andinclude different hardware than the other network processor 1900 a-n forcomparing data as it is transferred through a plurality of communicationnodes (e.g., a router or switch) and types of links.

The embodiment depicted in FIG. 19 can also be used for load balancing.For example, in the instance that the FIFO memory buffers 1925 a-bcoupled to network processor 1900 a are becoming full, but FIFO memorybuffers 1925 c-n have additional capacity that they can receive, thenetwork processor 1900 a-n can coordinate transfer of data from output1945 a of distribution module 1905 a to input 1950 n of distributionmodule 1905 n. In this manner the load differential can be balancedbetween the FIFO memory buffers 1925 a-b coupled to distribution module1905 a and the FIFO memory buffers 1925 c-n coupled to distributionmodule 1905 n. Load balancing can be conducted at any scale. Data can berouted from one distribution module to another in a series fashion suchthat the network data is received down stream at the desired analysisprocessor by being passed from one distribution module to another.

According to example embodiments of the present invention, the functionsof each network processor can also be divided between multipleprocessors as well as multiple logic devices. Front-end diversion,preparation of network data, and analysis using a logic device may alsobe implemented.

Example embodiments of the present invention can also include additionalfront-end diversion of data by additional logic devices, or by othermeans. For example, a programmable logic device (PLD) such as a FPGA canbe implemented to further divert the network data stream into multiplenetwork processors. The functions of example embodiments of the presentinvention can also be divided between several different devices in manydifferent configurations. For example, a FPGA, or a processor, canassign identification and perform the S/D/Q LUT manager functions; andany number of network processors, or other processors, can share thefilter manager and path manager functions as well as other functionsdescribed herein.

For example, referring now to FIG. 20, a system 2000 for analyzing anetwork data stream is shown according to an example embodiment of thepresent invention. Network data can be received by a FPGA 2005. The FPGA2005 can include a S/D/Q parser 2010 that extracts S/D/Q informationfrom a packet of network data and forwards the information to a S/D/QLUT manager 2015 that accesses a S/D/Q LUT 2020 and assigns a TID to thepacket. The TID can be sent to network processor path manager 2025 thataccesses a network processor path LUT 2030 and directs the packet to atleast one of the network processors 2035 a-n. An interleaver 2045 caninterleave the TID with a TS received from a TS counter 2040. Theinterleaved TID and TS can be transmitted to the appropriate networkprocessor 2035 a-n along with the packet of data.

The appropriate network processor 2035 a-n receives the interleavedTID/TS and network data. The interleaved TID/TS and network data isforwarded to at least one path manager 2065 a-n. The path manager 2065a-n receiving the network data accesses a path LUT 2055 a-n andidentifies an appropriate FIFO memory buffer 2060 a-n and analysisprocessor 2065 a-n assigned to the TID for receiving and analyzing thenetwork data along with other network data belonging to the sametransaction. A path control parser 2070 a-n can communicate with thepath manager 2065 a-n to enable and select channels of a distributionmodule 2080 so as to route the network data to the appropriate FIFOmemory buffer 2060 a-n and analysis processor 2065 a-n.

The appropriate FIFO memory buffer 2060 a-n can receive the network dataand act as a data buffer allowing for the corresponding analysisprocessor 2065 a-n to access and analyze the network data in turn.Results of the analysis or other data can be stored in a database or aHDD 2085 a-n. Each analysis processor 2065 a-n can be coupled to aplurality of HDDs 2085 a-n. HDDs are generally fairly cheap and canstore a relatively large amount of data. The speed of accessinginformation stored on a HDD can also make it advantageous to havemultiple HDDs coupled to a single analysis processor, such as analysisprocessor 2085 n, which is coupled to four HDDs 2085 n. For example,where a single analysis processor is coupled to five 200 gigabyte HDDsthe analysis processor has access to store and receive a terabyte ofdata. These HDDs can be configured in any fashion, for example accordingto any RAID standard.

According to example embodiments of the present invention, a networkprocessor apparatus can include multiple distribution modules coupled inseries and/or coupled in parallel to a network processor. For example,FIG. 21 illustrates an example embodiment of the present invention wherea single network processor 2100 is coupled to multiple distributionmodules 2105 a-n. The network processor 2100 receives network datarepresenting at least a portion of a data stream transmitted between asource 2110 and a destination 2115 in a network 2120. A firstdistribution module 2105 a receives at least a portion of the networkdata from the network processor 2100 along with a first control signal.Similarly, a second distribution module 2105 n receives at least aportion of the network data from the network processor 2100 along with asecond control signal. The distribution modules 2105 a-n can be coupledto multiple FIFO memory buffers 2125 a-n and analysis processors 2130a-n for analyzing the network data. In this manner analysis of thenetwork data can be distributed across multiple parallel orienteddistribution modules 2105 a-n and associated analysis processors 2130a-n from a single network processor 2100.

Referring now to FIG. 22 an example embodiment of the present inventionis shown where a single network processor 2200 receives network datarepresenting at least a portion of a data stream transmitted between asource 2205 and a destination 2210 in a network 2215. A firstdistribution module 2220 a can receive the network data and a controlsignal from the network processor 2200 and route the network data to anyof a plurality of memory buffers 2225 a-b and corresponding analysisprocessors 2230 a-n and/or a second distribution module 2220 n coupledto an output of the first distribution module 2220 a. The seconddistribution module 2220 n can receive the network data from the firstdistribution module 2220 a in the instance that the network dataprovided to the first distribution module 2220 a is routed to the outputcoupled to the second distribution module 2220 n. The seconddistribution module 2220 n can receive a second control signal from thenetwork processor 2200 and route the network data to an appropriate FIFOmemory buffer 2225 c-n and analysis processor 2330 c-n based on thesecond control signal. In this manner, analysis of the network data canbe distributed across multiple analysis processors 2230 a-n coupled tomultiple distribution modules 2220 a-n oriented in series from a singlenetwork processor.

Different modules containing different combinations of different aspectsof the present invention can be designed in a single analysis system, orin an overall analysis scheme. An analysis scheme can implement manydifferent levels of analysis for different communication links in asingle network or multiple networks depending on the level of concernregarding the particular link, or links. An analysis scheme or systemcan include two or more modules describing a set of parametersimplementing different aspects of the present invention at differentlevels. For example, in the instance that an analysis scheme or systemincludes three modules, for example a high level module, a medium levelmodule, and a low level module, different aspects of the presentinvention can be combined in different levels as desired.

A high level module can include, for example, a high level ofdemultiplexing, scaling, and a high level of processing bandwidth. Thehigh level module can implement hardware designed to handle such a largeamount of data and processing bandwidth as described in many embodimentsherein. The high level module can analyze the network data at manylayers of analysis and implement a low level of filtering andprioritized analysis. The high level module can analyze data using manytests at many layers of analysis at or approaching real-time speeds toinsure that as many errors as possible can be detected immediately, oras soon as possible.

A middle level module, can include, for example, a lower level ofdemultiplexing and scaling and can include a lower processing bandwidththan the high level module. The middle level module can implementfiltering and prioritized analysis to allow for a lower level ofprocessing bandwidth to process the most critical data using the mostcritical tests, but exclude lower priority tests and data from analysis.The middle level module can also selectively store data in a HDD forlater analysis. In this manner, the middle level module can analyzecertain data and perform certain tests at or approaching real timespeed, but allow analysis of other data at a later time, or not at all.

A low level module, for example, can include a lower level ofdemultiplexing (or no demultiplexing) and can include a lower processingbandwidth than the middle level module. For example, the low levelmodule can simply stream data to a HDD for later analysis. The low levelmodule can store all data related to a particular link and analyze thenetwork data when the analysis processors used for the middle and highlevel modules are no longer needed to analyze data at their higher levelof concern. Thus, many different combinations of any of the aspects ofthe present invention can be combined into modules that providedifferent levels of analysis in an overall analysis scheme or system.

The present invention may be embodied in other specific forms withoutdeparting from its spirit or essential characteristics. Combinations ofdifferent aspects of the present invention such as, but not limited todemultiplexing of network data so that the network data can be sortedbetween and analyzed by multiple analysis processors, distributing apiece of network data across multiple processors for network analysis,intelligently filtering network data so as to reduce the amount ofprocessing power required by excluding network data such as repetitivedata or data with known analysis results from further analysis,intelligently prioritizing different data analysis tests and algorithmsso that less critical tests, tests that have already been conducted,and/or tests with known results can be excluded for the sake of morecritical tests, and scaling various aspects of the present invention soas to remove bottlenecks in network analysis apparatuses can be embodiedin various configurations, sequences, and combinations.

At least a portion of some of the embodiments of the present inventionmay comprise a special purpose or general-purpose computer, processor,or logic device including various computer hardware and devices, asdiscussed in greater detail herein. Embodiments within the scope of thepresent invention can also include computer-readable media for carryingor having computer-executable instructions or data structures storedthereon. Such computer-readable media can be any available media thatcan be accessed by a general purpose or special purpose computer,processor or logic device. By way of example, and not limitation, suchcomputer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or otheroptical disk storage, magnetic disk storage or other magnetic storagedevices, or any other medium which can be used to carry or store desiredprogram code means in the form of computer-executable instructions ordata structures and which can be accessed by a general purpose computer,special purpose computer, or other logic device. When information istransferred or provided over a network or another communicationconnection (either hardwired, wireless, or a combination of hardwired orwireless) to a computer, the computer properly views the connection as acomputer-readable medium. Thus, any such connection is properly termed acomputer-readable medium. Various combinations of the above should alsobe included within the scope of computer-readable media.Computer-executable instructions comprise, for example, instructions,logic, and data which cause a general purpose computer, special purposecomputer, or special purpose processing device to perform a certainfunction or group of functions.

Combinations of these and other aspects of the present invention arealso encompassed within the scope of following disclosure, including theclaims that follow. The described embodiments are to be considered inall respects only as illustrative and not restrictive. The scope of theinvention is, therefore, indicated by the appended claims rather than bythe foregoing description. All changes which come within the meaning andrange of equivalency of the claims are to be embraced within theirscope.

1. A method for performing protocol analysis on network data, the methodcomprising: receiving network data representing at least a portion of adata stream transmitted in a network at a network processor;distributing the network data to a plurality of memory buffers connectedwith a plurality of analysis processors such that each packet of aparticular transaction is distributed to the same analysis processor,wherein each memory buffer generates a status signal indicating afullness of the memory buffer; assigning a priority to each of aplurality of protocol analysis tests; and performing selected protocolanalysis tests based on the priority when the status signal is above athreshold level.
 2. The method of claim 1, wherein the priority of eachprotocol analysis test is assigned based on at least one of: (i) whethereach protocol analysis test is a functional test, (ii) whether eachprotocol analysis test is a performance test, (iii) whether eachprotocol analysis test has been conducted in the past, (iv) whether eachprotocol analysis test has been conduct on a particular transaction inthe past, (v) and whether each protocol analysis test has identifiederrors for previous network data analyzed by the plurality of analysisprocessors.
 3. The method of claim 1, further comprising looking upentries in a priority look up table, each entry assigning a priority toat least two of the protocol analysis tests.
 4. The method of claim 1,wherein performing selected protocol analysis tests based on thepriority when the status signal is above a threshold level is performedat real-time speed.
 5. The method of claim 1, further comprisingcomparing the status signal to the threshold level to determine whetherthe status signal is above the threshold level, wherein the thresholdlevel represents an amount of data in a memory buffer.
 6. The method ofclaim 1, further comprising presenting a user a description of certainprotocol analysis tests not conducted on the network data due to thepriority of the certain protocol analysis tests.
 7. The method of claim1, further comprising presenting results of the selected protocolanalysis tests to the user.
 8. The method of claim 11, furthercomprising prioritizing tests of transaction identifications that havenot been conducted over transaction identifications that have beentested.
 9. The method of claim 15, further comprising prioritizingfunctionality protocol analysis tests over performance protocol analysistests.
 10. A protocol analyzer for performing protocol analysis ofnetwork data, the protocol analyzer comprising: a first networkprocessor configured to receive first network data representing at leasta portion of a data stream transferred in a first network link; a secondnetwork processor configured to receive second network data representingat least a portion of a data stream transferred in a second networklink; first analysis processors coupled to the first network processorfor performing protocol analysis tests on at least the first networkdata; second analysis processors coupled to the second network processorfor performing protocol analysis tests on at least the second networkdata, wherein the second analysis processors are configured tocommunicate with the first analysis processors; a first distributionmodule that distributes packets from the first network processor to thefirst analysis processors; and a second distribution module thatdistributes packets from the second network processor to the secondanalysis processors, wherein the second distribution module communicateswith the first distribution module such that packets from the firstdistribution module can be distributed to the second analysisprocessors.
 11. The protocol analyzer of claim 10, further comprising aprogrammable logic device (PLD) configured to receive network data froma network link and selectively direct the network data to at least oneof the first network processor and the second network processor.
 12. Theprotocol analyzer of claim 11, wherein the PLD is a Field programmablegate array (FPGA).
 13. The protocol analyzer of claim 11, furthercomprising first memory buffers that queue the first network data forthe first analysis processors and second memory buffers that queue thesecond network data for the second analysis processors, wherein thedistribution of the first network data and of the second network data iscontrolled by control signals generated by at least one of the firstnetwork processor and the second network processor such that packets inthe first network data and the second network data belonging to the sametransaction are distributed to the same analysis processor included ineither the first or second analysis processors.
 14. The protocolanalyzer of claim 11, wherein the PLD selectively directs the networkdata between the first and second network processors based on a statussignal received from at least one memory buffer coupled to one of thefirst or second analysis processors.
 15. The protocol analyzer of claim11, wherein the PLD includes a transaction look up table manager forassigning a transaction identification to the packets in the first andsecond network data.
 16. The protocol analyzer of claim 15, wherein eachnetwork processor includes an analysis processor LUT path manager forrouting the first or second network data to a particular analysisprocessor based on the transaction identification assigned to the firstor second network data by the PLD.
 17. The protocol analyzer of claim11, wherein an output of the first distribution module is coupled to aninput of the second distribution module.
 18. The protocol analyzer ofclaim 9, wherein the first network processor is coupled to the secondnetwork processor for providing communication between the first andsecond network processors.